GAO 


United  States  General  Accounting  Office _ 

Report  to  Congressional  Requesters 


ENVIRONMENTAL 

PROTECTION 

AGENCY 

Use  of  Precautionary 
Assumptions  in  Health 
Risk  Assessments  and 
Benefits  Estimates 


DISTRIBUTION  STATEMENT  A 

Approved  for  Public  Release 
Distribution  Unlimited 


20001026  060 


A  O 


Accountability  *  Integrity  *  Reliability 


'tone  QTFMinr  mmmmm  4 


GAO-01-55 


Contents 


Letter 


Appendixes 


Appendix  I:  Objectives,  Scope,  and  Methodology 

Appendix  II:  Dose-Response  Relationships  Based  on  Varying 
Assumptions 

Appendix  III:  Summary  of  EPA’s  1997  Particulate  Matter 
Standards 

Appendix  IV:  Issues  Associated  With  EPA’s  Setting  of  the  1997 
Particulate  Matter  Standards 


Tables 


Table  1:  1997  Particulate  Matter  Standards 


Figures 


Figure  1 :  Typical  Sequence  of  Risk  Assessment  and  Risk 
Management  Processes 

Figure  2:  EPA’s  Standard  (Default)  Dose-Response  Relationship 
for  Carcinogens 

Figure  3:  Dose-Response  Relationships  Based  on  Varying 
Assumptions 


Abbreviations 

EPA  Environmental  Protection  Agency 


Page  1 


GAO-Ol-55  EPA’s  Use  of  Precautionary  Assumptions 


Page  2 


GAO-Ol  55  EPA’s  Use  of  Precautionary  Assumptions 


i 

jj&.  GAO 

Jl^jj^^^^Accountability  » Integrity  *  Reliability _ 

United  States  General  Accounting  Office 
Washington,  D.C.  20548 


October  16,  2000 

The  Honorable  Bud  Shuster 
Chairman 

Committee  on  Transportation 
and  Infrastructure 
House  of  Representatives 

The  Honorable  Frank  R.  Lautenberg 
United  States  Senate 

Some  of  the  Environmental  Protection  Agency’s  (EPA)  regulations  set 
standards  that  limit  environmental  contaminants1  to  levels  that  are 
determined,  in  large  part,  on  the  basis  of  the  health  risks  they  pose.  For 
example,  EPA  sets  health-based  air  quality  standards  under  the  Clean  Air 
Act.  For  such  actions,  EPA  also  estimates  the  benefits  of  the  health-based 
standards.2  These  benefits  primarily  represent  the  estimated  dollar  value  of 
reductions  in  assessed  risks  to  human  health — illnesses  and  deaths  avoided 
as  a  result  of  decreased  pollution.3  However,  when  EPA  assesses  the  health 
risks  of  contaminants,  the  agency  is  faced  with  uncertainties  and  gaps  in 
scientific  knowledge  and  data.  As  a  result,  EPA’s  risk  assessments  include 
assumptions  about  the  relationship  between  specific  contaminants  and 
health  effects,  some  of  which  are  precautionary — that  is,  they  are  intended 
to  ensure  that  the  agency  does  not  underestimate  health  risks.  But  using 
such  precautionary  assumptions  to  estimate  benefits  could  produce  overly 
optimistic  estimates  of  the  benefits  of  regulatory  actions. 

Because  of  concerns  about  the  potential  impact  of  precautionary 
assumptions  on  benefits  estimates,  you  asked  us  to  examine  whether  EPAs 
benefits  estimates  for  major  environmental  regulations  that  establish 
health-based  standards  reflect  precautionary  assumptions  about  health 
risks.  As  agreed  with  your  offices,  this  report  identifies  (1)  key  factors  that 


fin  this  report,  we  use  the  term  contaminants  to  refer  to  substances  that  harm  human  health. 
EPA  uses  this  and  other  terms,  including  toxins,  toxic  substances,  and  pollutants,  to 
describe  these  substances. 

2Analyses  of  benefits  and  costs  are  required  for  all  regulations  that  are  “economically 
significant”  and  includes  those  expected  to  have  an  annual  impact  on  the  economy  of  $100 
million  or  more.  These  are  also  referred  to  as  major  rules  or  major  regulations. 

3Using  the  concept  of  the  “value  of  a  statistical  life,”  economists  have  developed  several 
methods  to  estimate  a  value  of  reductions  in  mortality  risk. 
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Results  in  Brief 


explain  why  EPA  uses  precautionary  assumptions  in  assessing  health  risks 
and  (2)  whether  EPA  used  and  identified  precautionary  assumptions  in 
estimating  the  health  risks  and  benefits  of  recent  major  regulations  setting 
health-based  standards. 

In  recent  years,  EPA  has  proposed  or  finalized  a  small  number  of  major 
regulations  establishing  health-based  standards,  and  these  have  been 
issued  under  two  statutes — the  Clean  Air  Act  and  the  Safe  Drinking  Water 
Act.4  To  illustrate  EPAs  use  of  precautionary  assumptions  in  setting  health- 
based  standards  and  estimating  their  benefits,  we  reviewed  two  of  these 
regulations:  (1)  air  quality  standards  for  particulate  matter  (commonly 
called  soot)  and  (2)  drinking  water  standards  for  arsenic.  As  a  result,  our 
findings  on  these  two  rules  may  not  be  generalized  to  all  of  EPA’s  major 
regulations  that  set  health-based  standards.  Consistent  with  our  objectives, 
we  did  not  review  the  economic  aspects  of  EPA’s  benefits  estimates,  such 
as  the  methods  used  to  estimate  dollar  values  for  lives  saved  and  illnesses 
avoided,  or  the  extent  of  EPAs  compliance  with  guidance  from  the  Office  of 
Management  and  Budget  and  EPA  on  the  preparation  of  benefits  estimates. 
(See  app.  I  for  a  detailed  discussion  of  our  scope  and  methodology.)  We  are 
also  reviewing  the  health  risk  assessment  procedures  and  assumptions  of 
four  agencies,  including  EPA,  and  plan  to  provide  the  results  of  this  review 
in  a  forthcoming  report  to  the  Chairman,  House  Committee  on  Commerce. 


Three  key  factors  influence  EPAs  use  of  precautionary  assumptions  in 
assessing  health  risks.  First,  EPA  is  influenced  by  its  mission  to  protect 
human  health  and  safeguard  the  natural  environment.  For  example,  in 
some  instances,  environmental  statutes  require  EPA  to  protect  the  public 
health  with  an  “adequate  margin  of  safety.”  Second,  EPA  is  influenced  by 
the  nature  and  extent  of  relevant  data — in  particular,  whether  studies  of  a 
contaminant’s  effects  on  people  are  available  or  whether  the  agency  must 
extrapolate  from  studies  using  other  animal  species.  Finally,  EPA  is 
influenced  by  the  nature  of  the  health  risk  being  evaluated,  such  as  whether 
the  contaminant  is  suspected  of  causing  cancer. 

The  two  regulations  that  we  examined  differed  in  the  extent  to  which  they 
used  precautionary  assumptions  in  estimating  health  risks  and  benefits. 


According  to  EPA,  the  majority  of  its  major  regulations  in  recent  years  have  established 
technology-based  or  performance-based  standards  that  reduce  pollution  using  available 
pollution  control  technology. 
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EPA  generally  did  not  rely  on  precautionary  assumptions  in  estimating 
either  the  health  risks  or  the  benefits  of  the  particulate  matter  regulation 
because  the  agency  had  better  data  than  is  often  the  case.  In  contrast, 
because  of  scientific  uncertainties  and  data  gaps,  EPA  relied  on  several 
precautionary  assumptions  in  estimating  the  health  risks  and  the  benefits 
of  the  arsenic  regulation.  As  a  result,  EPA’s  estimates  of  the  risk  of  bladder 
cancer  associated  with  arsenic  in  drinking  water  and  the  related  benefits  of 
the  proposed  rule  may  be  overstated  for  this  health  risk.  However,  EPA 
used  new  analytical  techniques  in  estimating  the  health  risks  and  benefits 
that  removed  certain  precautionary  assumptions  used  in  the  past,  thereby 
reducing  the  extent  to  which  the  benefits  of  reductions  in  bladder  cancer 
may  be  overstated.  In  its  proposed  arsenic  regulation,  EPA  identified  the 
key  health  uncertainties  and  precautionary  assumptions  it  used  in 
assessing  the  risks  of  arsenic  in  drinking  water  but  was  less  complete  in 
identifying  them  in  its  formal  cost-benefit  analysis.  Furthermore,  although 
its  guidance  on  cost-benefit  analyses  calls  for  assessing  uncertainty  using 
sensitivity  analysis,  EPA  did  not  perform  sensitivity  analysis  that  could 
have  shown  how  the  estimated  benefits  would  change  depending  upon  the 
health  assumptions  used.  We  are  recommending  that,  in  developing  its  final 
rule  on  arsenic,  EPA  fully  disclose  and  analyze  the  impact  of  the  key 
precautionary  health  assumptions  used  in  its  benefits  estimate. 
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Background 


EPA  assesses  human  health  risks  in  the  context  of  great  uncertainty  both 
in  terms  of  scientific  theory  and  data — about  the  adverse  effects  on  human 
health  posed  by  a  wide  variety  of  environmental  contaminants,  the 
relationship  between  toxicity  and  dose,  and  the  extent  of  people’s  exposure 
to  contaminants.  As  a  result,  EPA  must  use  assumptions,  based  on  general 
scientific  knowledge  and  policy  judgments,  when  it  lacks  more  specific 
scientific  knowledge  and  data  needed  to  assess  the  health  risks  posed  by 
contaminants.  The  National  Research  Council  of  the  National  Academy  of 
Sciences5  has  noted  that  risk  assessment  inevitably  includes  policy 
judgments  as  well  as  science.6 

EPA  has  issued,  over  the  past  25  years,  a  series  of  guidance  documents  that 
describe  the  principles,  policies,  and  practices  the  agency  employs  in 
evaluating  the  health  risks  (toxicity)  of  environmental  contaminants.  EPAs 
guidelines  cover  many  topics,  including  cancer,  reproductive  and 
developmental  toxicity,  neurotoxicity,  exposure  assessment,  and 
mutagenicity,  that  is,  the  capacity  to  cause  sudden  change  in  the  genetic 
material  of  a  cell.  EPA  personnel  who  conduct  risk  assessments  are  to  use 
EPAs  guidelines  to  ensure  consistency  in  the  interpretation  of  scientific 
information  across  all  of  the  agency’s  programs  and  regions. 

EPAs  risk  assessment  guidelines  set  forth  “default”  assumptions— generic 
approaches  based  on  general  scientific  knowledge  and  policy  judgment 
that  are  applied  to  various  elements  of  the  risk  assessment  process  when 
specific  scientific  information  is  not  available.  In  this  report,  we  refer  to  the 
default  assumptions  that  are  intended  to  avoid  underestimating  risk  as 
precautionary  assumptions.  There  is  an  ongoing  scientific  and  policy 
debate  concerning  whether  and  under  what  circumstances  some  of  the 
assumptions  used  in  estimating  human  health  risks  are  precautionary. 
According  to  some  analysts,  using  a  series  of  protective  default 
assumptions  within  the  same  risk  assessment  might  produce  results  that 
seriously  overstate  actual  risks.  Other  analysts  believe  that  the  degree  of 
precaution  may  not  be  as  great  as  some  argue — or  that  the  risk  assessment 


5The  National  Academy  of  Sciences,  a  private,  nonprofit  organization  composed  of  scholars, 
is  engaged  in  scientific  and  engineering  research  to  further  knowledge  and  advise  the 
federal  government.  The  National  Research  Council,  the  principal  operating  agency  for  the 
National  Academy,  provides  services  to  the  government,  the  public,  and  the  scientific  and 
engineering  communities. 

6See  Risk  Assessment  in  the  Federal  Government:  Managing  the  Process  (1983)  and  Science 
and  Judgment  in  Risk  Assessment  (1994). 
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methodologies  may  in  some  cases  actually  underestimate  risks — and  that  it 
is  appropriate  to  be  precautionary  about  health  risks. 

EPAs  practice  of  including  precautionary  assumptions  in  its  risk 
assessment  policies  and  practices  has  been  recognized  and  affirmed  by  the 
National  Research  Council.  According  to  the  Council,  EPAs  risk 
assessment  practices  rely  heavily  on  default  options  or  generic  approaches. 
The  Council  stated  that  these  default  options,  or  assumptions,  are  for  the 
most  part  chosen  to  “lead  to  risk  estimates  that,  although  plausible,  are 
believed  to  be  more  likely  to  overestimate  than  to  underestimate  the  risk  to 
human  health  and  the  environment.”7  The  Council  said,  however,  that  the 
choice  of  default  assumptions  should  have  a  decreasing  impact  on 
regulatory  decision-making  over  time  because,  as  scientific  knowledge 
increases,  uncertainty  diminishes  and  risk  assessments  should  be  less 
dependent  on  such  assumptions. 

Risk  assessments  of  chemicals  examine  the  types  of  adverse  health  effects 
that  might  occur  in  humans  and  wildlife  following  chemical  exposure 
(hazard  identification) ,  how  the  effects  vary  with  the  degree  of  exposure 
(dose-response  assessment),  and  the  degree  to  which  exposure  actually 
occurs  (exposure  assessment) .  Combining  this  information  enables  the 
overall  risk  to  be  described  for  decisionmakers  (risk  characterization) . 
Once  the  risk  is  characterized,  risk  management  involves  deciding  what 
actions,  if  any,  are  needed  to  prevent  or  reduce  the  risk,  such  as  limiting 
pollutant  emissions. 

As  shown  in  figure  1 ,  the  risk  management  decision  considers  other 
information  in  addition  to  the  risk  characterization.  For  example,  the  risk 
management  decision  may  be  affected  by  control  options — that  is,  the 
technologies  that  are  available  to  implement  a  standard — legal 
considerations,  and  economic  factors.  Economic  information  that  may  be 
considered  includes  cost-benefit  analyses,  which  are  required  for 
economically  significant  regulations.8  EPAs  regulations  that  establish 
national  health-based  standards,  such  as  drinking  water  standards  under 
the  Safe  Drinking  Water  Act,  typically  have  a  significant  effect  on  the 


7See  Science  and  Judgment  in  Risk  Assessment  (1994). 

Executive  Order  12866  requires  detailed  cost-benefit  analyses  for  all  economically 
significant  regulations  that  include  those  expected  to  have  an  annual  impact  on  the 
economy  of  $100  million  or  more.  EPA  is  also  directed  by  the  Unfunded  Mandates  Reform 
Act  of  1995  to  conduct  regulatory  cost-benefit  analyses  under  certain  circumstances. 
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economy.  Therefore,  EPA  conducts  both  a  risk  assessment  and  a  national 
benefits  estimate  in  these  cases,  while  in  many  other  cases  the  agency 
conducts  a  risk  assessment  but  conducts  only  a  limited  benefits  estimate  or 
none. 


Figure  1:  Typical  Sequence  of  Risk  Assessment  and  Risk  Management  Processes 


Risk  Risk 

assessment  management 


Source:  Research  and  Development:  Fiscal  Years  1997-1998  Research  Accomplishments,  EPA  (Dec. 
1999). 

EPA’s  various  program  offices,  including  those  responsible  for  pesticides, 
toxic  substances,  and  air  and  water  pollution,  conduct  many  health  risk 
assessments  that  vary  in  purpose  and  the  availability  of  data.  They  range 
from  single-purpose  screening  assessments  that  receive  limited  review  to 
fully  developed,  peer-reviewed  assessments  that  serve  as  the  basis  for 
major  regulations.  The  program  offices  may  use  the  hazard  identification 
and  dose-response  assessments  conducted  by  EPA’s  Office  of  Research  and 
Development.9  However,  the  program  offices  usually  conduct  the  exposure 
assessment  and  risk  management  phases  and  are  also  usually  responsible 
for  preparing  the  cost-benefit  analyses  for  major  rules. 


9Some  of  the  program  offices,  such  as  the  Office  of  Pesticide  Programs  and  the  Office  of 
Water,  do  all  or  some  of  their  own  hazard  identification  and  dose-response  assessments. 
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EPAs  methodologies  for  conducting  risk  assessments  and  benefits 
estimates  have  changed  in  recent  years  in  response  to  a  number  of  factors, 
including  recommendations  from  the  National  Research  Council,  changes 
in  the  agency’s  environmental  statutes,  the  availability  of  more 
sophisticated  mathematical  and  computer  models,  and  new  scientific  data 
on  and  increased  understanding  of  how  cancer  develops.  For  example,  in 
its  1994  report  on  EPAs  risk  assessment  activities,  the  National  Research 
Council  recommended  that  EPAs  risk  assessment  guidelines  identify  the 
specific  assumptions  that  are  default  options  and  clearly  state  the  scientific 
and  policy  basis  for  each  default  assumption  used.  EPA  has  proposed 
revisions  to  its  risk  assessment  guidelines  for  carcinogens  (cancer-causing 
substances)  that  call  for  identifying  the  default  assumptions  used  and  for 
highlighting  significant  issues;  they  also  provide  some  clarification  on 
departing  from  default  assumptions.  The  revised  guidelines  have  been  peer- 
reviewed  by  EPAs  Science  Advisory  Board  but  are  not  yet  final.  EPA  has 
started  to  incorporate  some  aspects  of  these  new  guidelines  into  some  risk 
assessments. 

Other  changes  include  guidance  on  EPAs  exposure  policies  and/or 
practices  aimed  at  reducing  the  use  of  some  precautionary  assumptions. 
For  instance,  EPAs  estimates  of  individuals’  exposures  to  contaminants 
can  be  precautionary  if  the  estimates  assume  that  individuals  are  exposed 
at  the  highest  levels.  Past  exposure  assessment  and  health  risk  assessment 
practices  at  EPA  have  sometimes  relied  on  exposure  estimates  derived 
from  a  hypothetical  “maximally  exposed  individual”  who  might  spend,  for 
example,  a  70-year  lifetime  drinking  only  groundwater  with  the  highest 
concentrations  of  contaminants  detected.  According  to  the  1997  report  of 
the  Presidential/Congressional  Commission  on  Risk  Assessment  and  Risk 
Management,  this  approach  was  often  based  on  such  unrealistic 
assumptions  that  using  it  impaired  the  scientific  credibility  of  risk 
assessments.  EPA,  like  other  federal  agencies,  has  moved  away  from 
exposure  assessments  relying  on  such  maximally  exposed  individuals.  For 
example,  EPAs  exposure  assessment  guidelines  have  adopted  the  use  of 
distributions  of  individual  exposures.  EPAs  current  guidance  indicates  that 
risk  assessments  should  include  both  central  estimates  of  exposure  (based 
on  either  the  mean  or  the  median  exposure)  and  estimates  of  the  exposures 
that  are  expected  to  occur  in  small,  but  definable,  “high-end”  segments  of 
the  population. 
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Three  Key  Factors 
Affect  EPA’s  Use  of 
Precautionary 
Assumptions 


Three  key  factors  principally  influence  EPA’s  use  of  precautionary 
assumptions  in  assessing  health  risks:  (1)  the  agency’s  mission  to  protect 
human  health  and  safeguard  the  environment;  (2)  the  nature  and  extent  of 
relevant  data-in  particular,  whether  studies  of  a  contaminant’s  effects  on 
humans  are  available  or  whether  the  agency  must  extrapolate  from  studies 
using  other  animal  species;  and  (3)  the  nature  of  the  health  risk  being 
evaluated,  such  as  whether  the  contaminant  is  thought  to  cause  cancer.  For 
example,  in  assessing  the  risks  of  contaminants  that  may  cause  cancer,  the 
agency  has  typically  made  the  precautionary  assumption  that  there  is  no 
safe  level  of  exposure— that  is,  that  any  exposure  poses  some  risk  of 
developing  cancer. 


EPAs  Mission  to  Protect 
Human  Health  and  the 
Environment 


EPA’s  mission,  articulated  in  its  strategic  plan  and  reflected  in  statutes, 
agency  policies,  and  practices,  is  to  protect  human  health  and  safeguard 
the  natural  environment.  This  mission  is  a  key  factor  encouraging  the 
agency  to  use  precautionary  health  risk  assumptions  in  the  absence  of 
convincing  scientific  knowledge.  For  example,  to  avoid  underestimating 
risks  to  human  health,  EPA  has  incorporated  a  number  of  precautionary 
assumptions  in  its  risk  assessment  guidelines  to  address  scientific 
uncertainties. 

In  some  instances,  environmental  statutes  require  EPA  to  protect  the 
public  health  with  a  margin  of  safety  either  in  assessing  risks  or  in  setting 
health-based  standards.  Under  the  1996  Food  Quality  Protection  Act,  for 
example,  EPA  is  required  to  give  special  consideration  to  children’s 
susceptibility  to  pesticide  residues  when  the  agency  sets  allowable  levels 
for  such  residues  in  food.  Among  other  things,  the  statute  requires  EPA  to 
make  precautionary  assumptions  in  its  risk  assessments  about  safe  levels 
of  pesticide  residues  for  children  when  data  are  incomplete  or  unreliable.10 
Under  the  Clean  Air  Act,  EPA  is  to  establish  national  standards  for  ambient 
(outdoor)  air  quality  to  protect  the  public  health  from  the  effects  of  certain 
widespread  air  pollutants,  such  as  carbon  monoxide  and  particulate  matter. 
The  act  requires  that  these  standards  be  set  at  levels  that  allow  for  an 
“adequate  margin  of  safety” 


10  Children  and  Pesticides:  New  Approach  to  Considering  Risk  Is  Partly  in  Place 
(GAO/HEHS-OO- 175,  Sept.  11,  2000). 
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The  Type  of  Data, 
Particularly  Whether 
Human  or  Animal  Studies 
Are  Available 


The  assumptions  that  EPA  uses  in  assessing  the  health  risks  of 
contaminants  depend  largely  on  whether  the  agency  has  adequate  studies 
of  their  effects  on  people,  known  as  epidemiological  studies,  or  only 
studies  of  other  animals  conducted  under  controlled  laboratory  conditions, 
called  toxicological  studies.  Epidemiologists  compare  two  or  more  groups 
of  people  to  determine  which  characteristics,  such  as  exposure  to 
contaminants,  distinguish  people  who  get  disease  from  people  who  do  not. 
Data  from  epidemiological  studies  are  preferred  for  characterizing  human 
health  risks  because  they  can  provide  the  most  direct  evidence  that  a 
substance  poses  health  risks  to  people.  If  these  studies  are  extensive  and  of 
good  quality,  EPA  generally  gives  greater  weight  to  epidemiological  data 
than  to  animal  data.  Nonetheless,  epidemiological  studies  require  the  use  of 
assumptions  to  address  uncertainties.  Among  the  key  assumptions  in  risk 
assessments  based  on  epidemiological  studies  that  may  be  precautionary 
are  the  following: 

•  Causality.  A  key  challenge  inherent  in  the  use  of  epidemiological  studies 
is  establishing  a  causal  relationship  between  the  contaminant  being 
assessed  and  the  identified  health  effect.  That  is,  epidemiological 
studies  may  show  that  a  particular  substance  is  associated  with  a  higher 
incidence  of  disease  in  an  exposed  population,  but  generally  the  studies 
do  not  provide  clear  evidence  that  the  substance  causes  the  disease. 
Furthermore,  other  scientific  information  that  would  help  establish  a 
causal  relationship — such  as  how  a  contaminant  causes  the  health 
effect,  referred  to  as  the  “mode  of  action” — often  does  not  exist.  In 
addition,  simultaneous  exposures  to  other  contaminants  can  reduce  the 
certainty  that  exposure  to  a  specific  contaminant  is  producing  the 
health  effect  that  has  been  identified.  Such  exposures,  referred  to  as 
confounding  factors,  can  only  be  recognized,  controlled,  and  measured 
to  a  certain  extent.  Because  it  is  difficult  to  establish  causation  on  the 
basis  of  epidemiological  evidence,  EPA  must  determine  whether  it  can 
infer  a  causal  relationship  between  exposure  to  a  contaminant  and 
observed  health  effects  through  its  review  of  the  available 
epidemiological  and  other  data. 

•  Extrapolation  from  high  doses.  The  populations  analyzed  in 
epidemiological  studies  may  be  exposed  to  doses  of  contaminants  that 
are  higher  than  the  doses  normally  occurring  in  the  environment.  For 
example,  epidemiologists  often  study  more  highly  exposed  populations, 
such  as  factory  workers.  As  a  result,  EPA  must  make  assumptions  when 
it  extrapolates  the  effects  of  high  doses  to  the  lower  dose  levels  to 
which  the  general  population  may  be  exposed.  These  assumptions  may 
or  may  not  be  precautionary. 
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•  Extrapolation  from  a  study  population  to  other  populations.  Using 
epidemiological  data  from  one  population  to  estimate  the  health  risks 
for  another  population  is  sometimes  precautionary.  For  example, 
epidemiological  data  on  the  health  risks  of  arsenic  have  been  developed 
principally  from  a  Taiwanese  population  whose  diets  may  make  them 
more  susceptible  than  the  U.S.  population  to  cancer  from  exposure  to 
arsenic.  EPA  used  the  Taiwanese  data  to  estimate  health  risks  in  the 
United  States  but  acknowledges  that  extrapolating  from  the  Taiwanese 
data  to  the  United  States  may  tend  to  overstate  the  risk  to  the  U.S. 
population.  On  the  other  hand,  extrapolating  from  other  epidemiological 
studies,  such  as  those  of  U.S.  workers,  may  tend  to  understate  some 
health  risks  for  the  U.S.  population  because  the  workers  in  the  studies 
would  not  include  individuals  with  higher  health  risks,  such  as  children, 
the  frail  elderly,  and  those  with  weakened  immune  systems  that  make 
them  more  susceptible  to  disease. 

For  most  contaminants,  epidemiological  studies  are  less  commonly 
available  than  animal  studies.  As  a  result,  EPA  relies  primarily  on  studies  of 
laboratory  animals  to  support  its  health  risk  assessments.11  Laboratory 
studies  of  animals  can  be  controlled,  and  thus  establishing  causation  is 
generally  not  an  issue.  Another  advantage  of  animal  studies  is  that  they  can 
provide  information  on  the  toxicity  of  contaminants  before  they  are  used, 
whereas  epidemiological  data  can  be  collected  only  after  human  exposure. 
When  using  these  toxicological  studies  to  assess  human  health  risks, 
however,  risk  assessors  must  rely  on  a  number  of  assumptions  that  may  be 
precautionary.  EPA’s  assumptions  relating  to  the  use  of  toxicological  data 
in  risk  assessment  include  the  following: 

•  Species-to-species  inference.  The  use  of  toxicological  studies  in 
assessing  health  risks  relies  on  the  assumption  that  laboratory  animals, 
such  as  rats,  mice,  and  monkeys,  are  surrogates  for  humans.  According 
to  the  National  Research  Council,  extrapolation  between  different 
species  is  supported  by  biological  principles  and  empirical  observations 
for  many  forms  of  biological  responses,  but  the  scientific  basis  of  such 
extrapolation  is  not  established  with  sufficient  rigor  to  allow  broad  and 
definitive  generalizations.  The  Council  has  also  stated  that  toxicity  is 
very  often  a  function  of  chemical  metabolism  and  that  differences 
among  animal  species  in  metabolic  handling  of  a  chemical  are  not 


11  According  to  EPA  officials,  a  growing  literature  suggests  that  cancer  health  risks  derived 
from  both  human  data  and  animal  data  are  similar. 
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uncommon.  The  Council  has  further  noted  that,  in  most  cases, 
information  on  a  chemical’s  metabolic  profile  in  humans  is  lacking  and 
identifying  the  animal  species  and  toxic  response  most  likely  to  predict 
the  human  response  accurately  is  generally  not  possible.  While  risk 
assessors  assume  that  health  effects  seen  in  laboratory  animals  are 
likely  to  be  seen  in  humans  as  well,  some  tests  on  laboratory  animals 
have  not  identified  human  health  problems.  Perhaps  the  best  known 
case  is  that  of  the  drug  thalidomide.  No  adverse  health  effects  were 
found  in  animal  testing;  however,  in  humans  it  caused  severe  birth 
defects  in  the  children  of  women  who  took  the  substance.  Similarly, 
while  epidemiological  studies  have  shown  that  arsenic  is  a  human 
carcinogen,  test  animals  have  not  developed  cancer  from  exposure  to 
arsenic. 

•  Extrapolation  from  high  doses  to  low  doses.  Animal  studies  must  use 
much  higher  doses  than  the  doses  that  people  are  typically  exposed  to 
because  millions  of  animals  would  have  to  be  exposed  to  low  doses  in 
order  to  detect  adverse  health  effects.  Although  some  critics  question 
the  validity  of  extrapolating  the  effects  of  high  doses  of  contaminants 
given  to  research  animals  to  low  doses  that  people  encounter  in  the 
environment,  the  National  Research  Council  recommended  that  EPA 
continue  to  assume  adverse  effects  from  lower  doses  in  the  absence  of 
other  information. 

•  Use  of  highly  sensitive  animal  species.  Toxicology  studies  often  use 
animals  that  are  highly  sensitive  to  the  contaminant  being  studied  in 
order  to  ensure  a  detectable  response.  Similarly,  when  there  are  multiple 
studies  assessing  the  toxicity  of  a  substance  but  information  is  lacking 
on  which  species  responds  most  like  humans,  EPA  uses  the  most 
sensitive  species  in  assessing  human  risk.12 

In  addition  to  assumptions  inherent  in  the  use  of  toxicological  data  that 
may  be  precautionary,  there  are  other  assumptions,  such  as  the  following, 
that  are  generally  seen  as  not  being  precautionary: 

•  Studies  account  for  exposure  to  only  one  chemical.  Animal  studies 
usually  address  an  individual  chemical,  even  though  people  are  often 
exposed  to  multiple  contaminants  in  the  environment.  There  is  evidence 
that  for  some  contaminants,  combinations  of  exposures  may  increase 
health  risks  to  higher  levels  than  would  be  estimated  by  simply  adding 
the  individual  risks  together.  Therefore,  a  toxicity  assessment  that  relies 


12 Reference  Dose  (RfD):  Description  and  Use  in  Health  Risk  Assessments,  EPA  (Mar.  1993). 
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on  animal  studies  that  address  a  single  contaminant  may  tend  to 
underestimate  toxicity  to  people  who  are  exposed  to  multiple 
contaminants. 

•  Studies  account  for  exposure  through  only  one  route.  Similarly,  people 
may  be  exposed  to  a  contaminant  by  more  than  one  route,  which  could 
mean  that  risks  assessed  on  the  basis  of  an  animal  study  using  only  one 
such  route  could  underestimate  the  risks  to  people. 

•  Studies  generally  use  mature  animals.  According  to  EPA  officials,  most 
toxicology  studies  use  mature  animals,  thereby  ignoring  effects  of 
exposure  on  the  developing  animal— which  may  be  more  frequent,  more 
severe,  or  very  different  in  nature  from  the  effects  on  mature  animals. 


The  Type  of  Health  Risk 
Being  Assessed 


EPA’s  Assessment  of  Cancer 
Risks 


Over  time,  EPA  has  developed  different  policies  and  methodologies  for 
assessing  different  types  of  health  risks.  Most  notably,  EPA’s  approach  for 
cancer  risk  assessment  has  differed  from  its  approaches  for  noncancer  risk 
assessment.  Both  types  of  risk  assessments  rely  on  assumptions,  including 
precautionary  assumptions;  however,  the  specific  assumptions  used  differ 
with  the  type  of  risk  being  assessed.  Central  to  the  development  of  EPA’s 
approach  to  risk  assessment  for  carcinogens  has  been  the  theory  that  even 
a  small  number  of  changes  in  a  single  cell  can  lead  to  the  uncontrolled 
growth  of  cells  known  as  cancer.  This  theory  implies  that  there  is  no  safe 
level  of  exposure,  or  threshold,  below  which  the  contaminant  does  not 
pose  a  risk.  In  contrast,  for  health  problems  other  than  cancer,  EPA  has 
generally  posited  that  there  is  some  safe  level  of  exposure  to  a  contaminant 
before  health  effects  occur. 

In  assessing  cancer  risks,  EPA  develops  a  quantitative  estimate  of  the 
expected  increase  in  the  incidence  of  cancer  resulting  from  varying 
exposures  to  a  contaminant.  This  estimate  is  called  the  dose-response 
relationship.  In  dose-response  assessment,  EPA  has  not  traditionally 
speculated  as  to  how  the  potential  carcinogen  induces  cancer,  and  such 
data  have  generally  not  been  available,  according  to  the  director  of  the 
quantitative  risk  methods  group  within  EPA’s  Office  of  Research  and 
Development.  In  developing  the  dose-response  relationship,  EPA  generally 
uses  two  key  precautionary  assumptions: 

•  A  linear,  no-threshold  relationship  between  the  dose  and  the  health 
effects  at  low  doses.  This  assumption  posits  no  safe  level  of  exposure  to 
a  carcinogen— that  is,  any  exposure  presents  some  risk  of  developing 
cancer.  However,  for  some  contaminants,  a  nonlinear  dose-response 
relationship  is  believed  to  exist,  while  other  contaminants  are  believed 
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to  have  a  threshold  (see  app.  II).  In  addition,  this  assumption  relies  on 
the  observed  relationship  between  high  doses  and  the  incidence  of 
cancer — based  on  either  epidemiological  or  toxicological  data — to 
predict  or  extrapolate  the  cancer  risk  at  the  much  lower  levels  for  which 
no  data  on  health  responses  are  available.  When  EPA  does  not  have 
sufficient  data  or  a  model  for  extrapolating  the  cancer  risk  at  lower 
doses,  EPAs  guidelines  for  cancer  risk  assessment  call  for  estimating  a 
linear  relationship  between  dose  and  health  effects.  That  is,  each 
additional  increment  of  exposure  is  assumed  to  produce  the  same 
proportional  change  in  the  health  effect. 

•  “  Upper  bound ”  estimates  of  risk.  EPA  uses  statistical  procedures  to 
develop  an  estimate  of  the  dose-response  relationship  that  is  very 
unlikely  to  be  exceeded  by  the  true  risk.  This  estimate,  typically  set  at 
the  95-percent  upper  confidence  limit,  represents  an  upper  bound  on 
risk  (see  fig.  2). 13  According  to  a  report  prepared  for  the  National 
Commission  on  Risk  Assessment  and  Risk  Management,14  if  the 
unknown  dose-response  relationship  is  linear  at  low  doses,  this 
procedure  overestimates  the  true  risk  by  a  relatively  small  factor, 
usually  two-  to  threefold.  If,  however,  the  true  relationship  at  low  doses 
is  nonlinear,  this  approach  will  overestimate  risks  by  larger  factors  that 
increase  as  the  dose  levels  decrease. 


13The  95-percent  upper  confidence  limit  is  a  statistically  derived  upper-limit  estimate  of  risk 
that  is  designed  to  overstate  rather  than  understate  human  risk. 

lAA  Survey  of  Methods  for  Chemical  Health  Risk  Assessment  Among  Federal  Regulatory 
Agencies,  prepared  for  the  National  Commission  on  Risk  Assessment  and  Risk  Management 
by  Lorenz  R.  Rhomberg,  Ph.D.,  Harvard  Center  for  Risk  Analysis,  Harvard  School  of  Public 
Health  (1996). 
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Figure  2:  EPA’s  Standard  (Default)  Dose-Response  Relationship  for  Carcinogens 
Increasing  response  (health  risk) 


Increasing  dose  (exposure) 


Note:  The  figure  above  is  for  a  hypothetical  contaminant.  The  relative  difference  between  the  95- 
percent  upper  confidence  limit  and  the  central  estimate  varies  across  contaminants. 

The  National  Research  Council  stated  in  1994  that  EPA  should  continue  to 
use  upper-bound  estimates  of  lifetime  cancer  risks.  It  noted,  however,  that 
whenever  possible,  this  estimate  should  be  supplemented  with  other 
descriptions  of  cancer  risk  that  more  fully  reflect  the  uncertainty 
associated  with  these  estimates. 

EPA’s  cancer  risk  assessment  guidelines,  issued  in  1986,  emphasize  that  its 
default  assumptions  lead  to  a  plausible  upper  limit  on  the  risk  that  is 
consistent  with  some  proposed  mechanisms  for  how  cancer  develops.  The 
guidance  also  states,  however,  that  the  estimate  is  not  necessarily  a 
realistic  predictor  of  risk,  since  the  true  value  of  the  risk  is  unknown  and 
may  be  as  low  as  zero. 

In  addition  to  the  precautionary  assumptions  that  relate  to  the  dose- 
response  relationship,  EPA’s  risk  assessments  for  cancer  reflect  other 
precautionary  assumptions  such  as  the  following: 
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•  A  substance  that  is  carcinogenic  in  animals  is  likely  to  be  a  human 
carcinogen. 

•  When  benign  and  malignant  tumors  are  observed  in  animals,  the 
incidence  of  both  is  combined  to  represent  the  substance’s  carcinogenic 
potential  in  humans. 

•  In  the  absence  of  information  indicating  which  species  responds  most 
like  humans,  the  animal  species  exhibiting  the  greatest  carcinogenic 
sensitivity  is  given  the  greatest  emphasis  in  developing  estimates  of 
human  cancer  risk. 

As  for  other  types  of  risk  assessment,  EPA’s  policies  and  practices  for 
assessing  cancer  risks  have  been  changing  as  new  analytical  techniques  are 
developed  and  scientific  knowledge  increases.  For  example,  the  agency’s 
guidelines  for  assessing  carcinogenic  risk  are  being  revised.  Among  the 
issues  addressed  in  the  draft  guidelines  is  the  no-threshold  assumption  for 
carcinogens.  As  more  research  into  the  mechanisms  of  how  cancer 
develops  has  become  available,  there  have  been  challenges  to  the  theory 
that  there  is  no  safe  level  of  exposure  to  a  carcinogen.  For  example,  in  1994 
the  National  Research  Council  reported  that  risk  models  that  use  a 
threshold  are  plausible  for  many  carcinogens. 
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In  March  2000,  a  federal  appeals  court  ruled  that  in  setting  a  health-based 
maximum  contaminant  level  goal  for  chloroform  (a  chemical  byproduct  of 
chlorination,  the  most  widely  used  technique  for  ensuring  the  safety  of 
drinking  water)  under  the  Safe  Drinking  Water  Act,  EPA  “openly  overrode 
the  ‘best  available’  scientific  evidence,”  which  suggested  that  there  is  a 
threshold  for  the  carcinogenic  effects  of  chloroform.15  In  1998,  EPA 
concluded  that  the  assumption  of  a  nonlinear  relationship,  which  is 
permitted  under  its  existing  carcinogenic  risk  guidelines  issued  in  1986, 
would  be  more  appropriate  than  a  linear  assumption.  However,  in  the  final 
rule,  EPA  used  a  no-threshold,  linear  assumption.  According  to  EPA 
officials,  staff  responsible  for  conducting  risk  assessments  have  been 
reluctant  to  depart  from  the  standard  cancer  defaults  in  EPA’s  existing 
policy  because  of  uncertainties  about  when  such  a  departure  would  be 
appropriate.16  The  criteria  for  departing  from  the  defaults  have  not  been 
clearly  articulated  in  the  past,  and  there  is  ongoing  debate  over  whether 
departing  would  be  protective  of  sensitive  populations,  including  children. 
The  revised  guidelines,  which  are  not  finalized  as  of  October  2000,  will 
offer  more  direction  to  risk  assessors  in  terms  of  when  and  how  to  depart 
from  the  traditional  no-threshold  assumption,  according  to  EPA  officials. 


15 Chlorine  Chemistry  Council  v.  Environmental  Protection  Agency  206  F.3d  1286,  1290  (D.C. 
Cir.  2000). 

16In  the  case  of  chloroform,  the  risk  assessors  departed  from  the  standard  cancer  default, 
but  EPA  made  the  policy  decision  to  use  the  standard  default,  in  part,  because  not  using  the 
default  would  represent  a  “significant  and  precedential”  application  of  new  science  that  had 
important  implications  for  other  contaminants  regulated  under  the  Safe  Drinking  Water  Act. 


Page  18 


GAO-Ol-55  EPA’s  Use  of  Precautionary  Assumptions 


EPA’s  Assessment  of  Noncancer 
Health  Risks 


In  addition  to  assessing  carcinogenic  risks,  EPA  assesses  other  threats  to 
human  health,  such  as  respiratory  problems  associated  with  air  pollution. 
Because  of  the  wide  variety  of  noncancer  health  effects  and  the  diversity  of 
ways  in  which  contaminants  are  suspected  of  working,  EPA  does  not  have 
a  single  approach  for  assessing  noncancer  risks  comparable  to  its 
agencywide  guidelines  for  assessing  cancer  risks.17  However,  one 
approach — using  what  is  called  a  reference  dose,  or  RfD — is  used  most 
frequently  to  determine  a  threshold  or  safe  level  of  exposure  to  a 
contaminant.  The  reference  dose  is  an  estimate  of  a  daily  exposure  level 
that  is  not  likely  to  cause  “appreciable  risk  of  deleterious  effects  during  a 
lifetime.”18 

In  estimating  a  safe  level  of  exposure  to  the  noncarcinogenic  effects  of  a 
contaminant,  EPA  first  determines  the  dose  level  at  which  no  adverse 
effects  have  been  observed.  This  level  is  then  reduced  because  of 
uncertainties  in  the  data — generally  toxicological  data  from  animal  studies. 
That  is,  the  dose  at  which  no  adverse  effects  have  been  observed  is  reduced 
(divided)  by  one  or  more  uncertainty  factors  (sometimes  called  safety 
factors).  The  uncertainty  factors  account  for  the  possibility  that  people 
might  need  a  lower  level  of  exposure  to  better  ensure  safety.19  Each 
uncertainty  factor  typically  reduces  the  level  at  which  no  adverse  effects 
have  been  observed  to  one-tenth  the  original  dose.  The  following  two 
uncertainty  factors  are  used  most  frequently: 

•  A  factor  of  10  is  generally  used  to  account  for  variation  in  sensitivity 
among  people,  such  as  the  elderly  and  other  populations  that  are  more 
susceptible  to  diseases. 

•  A  factor  of  up  to  10  is  generally  used  to  account  for  the  uncertainty 
associated  with  using  the  results  of  laboratory  animal  studies  to 
estimate  the  health  effects  expected  in  people.  This  factor,  usually  set  at 
10,  stems  from  the  concern  that  people  could  be  more  sensitive  to  the 
toxic  effects  of  a  contaminant  than  are  laboratory  animals.  For  example, 


17The  agency  has,  however,  issued  risk  assessment  guidelines  for  several  types  of  noncancer 
health  threats,  including  developmental  toxicity  and  reproductive  toxicity. 

18EPA  also  develops  reference  concentrations  (RfC),  estimates  of  a  daily  exposure  level  (in 
terms  of  air  concentrations  rather  than  dose)  that  is  likely  to  be  without  an  appreciable  risk 
of  adverse  noncancer  effects  during  a  lifetime. 

19EPA  uses  the  term  “uncertainty  factor”  rather  than  “safety  factor”  because  of  concerns  that 
the  latter  term  implies  an  absolutely  safe  level,  an  assurance  the  agency  does  not  believe  it 
can  provide. 
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small  animals  have  faster  metabolic  processes,  allowing  them  to 
eliminate  contaminants  from  their  bodies  more  quickly  than  people  can. 

Other  factors  may  be  added  to  account  for  such  uncertainties  as  a  safe  level 
for  lifetime  human  exposure  when  only  short-term  animal  studies  are 
available.  According  to  a  report  prepared  for  the  National  Commission  on 
Risk  Assessment  and  Risk  Management,  two  or  three  factors  are  typically 
used  in  assessing  noncancer  risks.  This  could  result  in  as  much  as  a  100-  or 
1,000-fold  decrease  in  the  estimated  safe  level  of  exposure. 

As  for  cancer  risk  assessment,  EPA’s  approaches  for  noncancer  risk 
assessment  are  changing  over  time  with  increases  in  scientific 
understanding  of  how  contaminants  cause  adverse  health  effects  and  the 
relationship  between  the  dose  of  a  contaminant  and  the  increased  health 
risk  expected.  In  some  instances,  EPA  has  enough  information  on  the 
general  human  population  to  estimate  a  quantitative  relationship  between 
exposures  to  varying  concentrations  of  the  contaminant  and  the  expected 
increased  health  risks.  In  these  cases,  EPA  does  not  have  to  use  the 
reference  dose  approach  that  relies  on  various  factors  to  reflect 
uncertainties  and  data  gaps.  Such  is  the  case  with  six  widespread  air 
pollutants  for  which  EPA  has  established  national  health-based  standards 
under  the  Clean  Air  Act,  including  particulate  matter. 


EPA’s  Use  and 
Disclosure  of 
Precautionary  Risk 
Assumptions  in 
Estimating  Benefits  for 
Two  Major  Regulations 


The  two  major  health-based  regulations  we  examined  differed  in  the  extent 
to  which  they  used  precautionary  assumptions  in  estimating  health  risks 
and  benefits.  EPA  used  such  assumptions  only  to  a  limited  extent  in  the 
case  of  the  1997  air  quality  standards  for  particulate  matter  but  relied  on 
them  more  in  its  recently  proposed  drinking  water  standard  for  arsenic. 
However,  in  assessing  the  health  risks  of  arsenic  and  estimating  the  related 
benefits  of  the  proposed  standard,  EPA  used  new  analytical  techniques  that 
removed  other  precautionary  assumptions  used  in  the  past.  In  its  proposed 
arsenic  regulation,  EPA  identified  the  key  health  uncertainties  and 
precautionary  assumptions  it  used  in  assessing  the  risks  of  arsenic  in 
drinking  water  but  was  less  complete  in  identifying  them  in  its  formal  cost- 
benefit  analysis.  Furthermore,  although  its  guidance  on  cost-benefit 
analyses  calls  for  assessing  uncertainty  using  sensitivity  analysis,  EPA  did 
not  perform  sensitivity  analysis  that  could  have  provided  information  on 
the  potential  impact  of  the  precautionary  health  assumptions  that  underlie 
its  benefits  estimate. 
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EPA’s  1997  National 
Ambient  Air  Quality 
Standards  for  Particulate 
Matter 


Under  the  Clean  Air  Act,  EPA  sets  health-based  National  Ambient  Air 
Quality  Standards  for  certain  widespread  air  pollutants  including 
particulate  matter— the  generic  name  for  a  mixture  of  air  pollutants 
commonly  found  across  the  United  States.20  EPA  is  required  to  review  the 
existing  standards  every  5  years  to  ensure  that  the  standards  are  based  on 
the  most  recent  scientific  information.  EPA  last  revised  the  particulate 
matter  standards  in  July  1997,  at  which  time  it  added  standards  for  fine 
particulate  matter  (particles  less  than  2.5  micrometers  in  diameter).21  (See 
app.  Ill  for  information  on  the  several  standards  EPA  set  for  particulate 
matter.)  EPA  estimated  that  the  new  standards  for  fine  particles  could  save 
between  3,300  and  15,600  lives  annually.  EPA’s  monetary  estimate  of  the 
health  benefits  of  the  new  standards,  which  included  not  only  lives  saved 
but  also  other  benefits,  such  as  cases  of  chronic  bronchitis  avoided,  ranged 
from  $14.5  billion  to  $96.1  billion  per  year.22 

When  EPA  evaluated  the  health  risks  from  particulate  matter  in  developing 
its  1997  standards,  it  generally  did  not  rely  on  precautionary  assumptions. 
This  was  largely  because  EPA  had  better  data  at  relevant  exposure  levels 
than  is  often  the  case  when  the  agency  assesses  risks.  The  strengths  of  the 
data  included  the  following: 

•  EPA  had  a  large  body  of  epidemiological  research  upon  which  to  base 
its  risk  assessment.  This  contrasts  with  most  of  EPA’s  risk  assessments, 
which  must  rely  on  animal  studies.  Thus,  EPA  did  not  have  to  make  the 
assumptions  needed  when  animal  studies  are  used  to  predict  human 
health  risks.  Moreover,  the  epidemiological  research  showed  largely 
consistent  associations  between  particulate  matter  in  the  outdoor  air 
and  a  variety  of  health  problems,  including  premature  death  from 
respiratory  and  cardiovascular  causes,  particularly  among  the  elderly; 
exacerbation  of  cardiopulmonary  and  respiratory  illnesses;  an  increased 


20Particulate  matter  can  include  many  chemically  and  physically  diverse  substances  and  can 
vary  significantly  by  location.  A  number  of  activities  typically  add  to  the  concentration  of 
particulate  matter  in  the  air,  including  combustion  from  power  plants,  industrial  facilities, 
cars,  trucks,  and  wood  stoves;  construction  and  demolition  activities;  and  road  dust.  Among 
the  major  chemical  components  of  particulate  matter  are  sulfates,  nitrates,  acids,  metal 
compounds,  and  water. 

21 A  micrometer,  also  known  as  a  micron,  is  one-millionth  of  a  meter. 

22The  particulate  matter  rule  accounts  for  as  much  as  54  percent  of  the  total  benefits  of  the 
48  major  rules  issued  governmentwide  from  Apr.  1,  1995,  to  Mar.  31, 1999,  for  which  federal 
agencies  estimated  benefits. 


Page  21 


GAO-Ol-55  EPA’s  Use  of  Precautionaiy  Assumptions 


incidence  of  bronchitis  in  children;  and  an  increased  number  of 
childhood  asthma  attacks. 

•  EPA  had  sufficient  data  to  estimate  a  range  of  risks  showing  how  risk 
would  vary  with  different  concentrations  of  particulate  matter  in  the  air. 
Typically  called  a  dose-response  relationship,  this  is  referred  to  as  a 
concentration-response  relationship  in  the  case  of  air  pollutants,  where 
the  observed  relationship  is  between  the  concentration  of  the  pollutant 
in  outdoor  air  and  the  given  health  effects.  In  contrast,  for  most  other 
noncarcinogenic  pollutants,  EPA  would  typically  make  only  one 
estimate — that  of  a  dose  likely  to  be  safe  (a  reference  dose  discussed 
above) .  In  estimating  a  dose  likely  to  be  safe,  EPA  would  typically 
reduce  the  allowable  exposure  by  applying  uncertainty  factors  to 
account  for  such  things  as  people’s  differing  susceptibilities  to  the 
pollutant’s  health  effects. 

•  Furthermore,  EPA  had  information  about  directly  relevant 
concentration  levels  of  particulate  matter  in  the  air,  that  is,  at  levels  to 
which  the  population  was  actually  exposed.  Thus,  the  agency  did  not 
have  to  extrapolate  exposure  information  from  studies  using  higher 
doses.  Because  the  studies  EPA  relied  on  were  based  on  U.S.  data  for  a 
number  of  urban  areas,  EPA  did  not  have  to  extrapolate  the  effects 
observed  in  one  population  to  estimate  the  health  effects  in  another 
population.  Moreover,  in  assessing  the  health  effects  of  particulate 
matter  on  the  basis  of  findings  from  numerous  epidemiological  studies, 
EPA  used  risk  levels  that  represented  central  tendency  estimates  rather 
than  upper-bound  estimates  of  the  health  effects. 

Despite  the  strengths  of  the  available  epidemiological  data,  there  were 
significant  scientific  uncertainties  in  EPAs  evaluation  of  particulate 
matter’s  health  effects,  which  led  EPA  to  use  at  least  one  precautionary 
assumption.  EPA’s  sensitivity  analyses  showed  that  the  most  significant 
uncertainty  was  whether  a  threshold  concentration  existed,  that  is, 
whether  the  health  effects  would  be  associated  with  particulate  matter  at 
any  level  of  exposure.  In  the  quantitative  health  risk  assessment  used  in  the 
standard-setting  process,  EPA  made  the  precautionary  assumption  that 
there  was  no  threshold  for  the  health  problems  associated  with  particulate 
matter.  EPA  officials  said  they  made  this  precautionary  assumption 
because  they  did  not  have  information  that  indicated  the  existence  of  a 
threshold  for  the  various  health  effects  included  in  the  risk  assessment. 

However,  EPAs  methodology  did  not  estimate  the  health  risks  down  to  a 
concentration  of  zero.  EPA  estimated  the  health  effects  associated  with 
particulate  matter  starting  at  the  larger  of 
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•  the  level  of  naturally  occurring  particulate  matter  (also  called  the 
background  level)23  or 

•  the  lowest  average  annual  level  of  particulate  matter  observed  in  the 
epidemiological  studies  EPA  relied  upon.24 

EPA’s  estimate  of  adverse  health  effects  was  lower  than  it  would  have  been 
had  the  agency  included  health  risks  at  the  lowest  levels.  However,  EPA 
officials  believe  it  was  appropriate  to  measure  health  effects  only  above  the 
naturally  occurring  levels  of  particulate  matter  because  it  is  unlikely  that 
concentrations  of  particulate  matter  could  be  reduced  below  such  levels. 
Also,  EPA  did  not  estimate  health  effects  below  the  levels  reported  in  the 
epidemiological  studies  because  of  the  uncertainties  about  health  effects  at 
such  levels. 


23EPA  made  the  following  “base  case”  assumptions  about  background  concentrations  of 
particulate  matter  (expressed  in  terms  of  micrograms  per  cubic  meter  of  air)  in  the  two 
locations  included  in  its  risk  analysis:  fine  particulate  matter  (also  called  PM2.5)— 3.5  in 
Philadelphia  County,  Pennsylvania,  and  2.5  in  Southeast  Los  Angeles  County,  California; 
course  and  fine  particulate  matter  (also  called  PM  10) — 8  in  Philadelphia  County  and  6  in 
Southeast  Los  Angeles  County.  EPA  also  conducted  sensitivity  analyses  looking  at  the 
effects  of  alternative  assumptions  about  background  levels. 

24For  example,  the  lowest  median  annual  concentration  of  particulate  matter  in  a  key  study 
assessing  the  mortality  risks  associated  with  this  pollutant  was  9  micrograms  per  cubic 
meter  of  air. 


Page  23 


GAO-Ol-55  EPA’s  Use  of  Precautionary  Assumptions 


When  EPA  estimated  the  health  benefits  of  the  particulate  matter  standards 
as  part  of  its  cost-benefit  analysis,  it  used  essentially  the  same  dose- 
response  relationships  used  in  the  risk  assessment.  EPA’s  cost-benefit 
analysis  also  identified  the  scientific  uncertainties  associated  with  this 
dose-response  relationship.  However,  in  estimating  the  benefits  of  this  rule, 
EPA  did  not  include  the  risk  assessment’s  precautionary  assumption  that 
there  was  no  threshold  (above  certain  levels)  for  the  health  effects.  Rather, 
EPA  estimated  benefits  using  two  different  assumptions  about  thresholds 
above  background  levels — at  concentrations  of  12  and  15  micrograms  of 
fine  particulate  matter  per  cubic  meter  of  air.  According  to  EPA  officials, 
the  additional  benefits  from  risk  reductions  at  concentrations  of  particulate 
matter  below  12  micrograms  per  cubic  meter  would  not  be  as  significant  as 
at  the  higher  levels  because  few  geographic  locations  would  achieve 
concentrations  at  the  lowest  pollution  levels.25 

Another  assumption  that  EPA  made  in  evaluating  particulate  matter’s 
health  risks  and  in  estimating  benefits,  an  assumption  that  is  typically  used 
when  epidemiological  data  are  involved,  was  causality — that  is,  EPA 
assumed  that  there  is  a  causal  relationship  between  particulate  matter  and 
the  health  effects  with  which  it  has  been  associated.  Some  scientists  and 
other  commenters  on  EPA’s  proposed  particulate  matter  regulation 
believed  that  the  causality  assumption  was  precautionary.  EPA,  however, 
did  not  characterize  this  assumption  as  precautionary  because  it  believed 
the  consistency  of  the  results  from  a  large  number  of  locations  and  the 
coherent  nature  of  the  results  suggest  a  likely  causal  role  of  particulate 
matter  in  contributing  to  these  health  effects. 

Nonetheless,  many  questions  remain  about  how  particulate  matter  may  be 
causing  premature  death  and  other  adverse  health  effects,  including 
questions  of  whether  there  may  be  confounding  agents,  such  as  other  air 
pollutants  (e.g.,  sulfur  dioxide  and  ozone),  that  may  be  causing  at  least 
some  of  the  problems.  The  National  Research  Council  has  cited  the 
“relatively  consistent  but  poorly  understood  associations  between  ambient 
particulate  matter  concentrations  and  various  adverse  health  effects.  26 
Because  of  the  many  unknowns,  EPA’s  Clean  Air  Scientific  Advisory 


25 As  a  result  of  guidance  from  its  economic  advisory  committee,  EPA’s  more  recent  benefits 
estimates  done  for  other  regulations,  such  as  its  estimate  of  the  prospective  benefits  of  the 
Clean  Air  Act,  have  assumed,  among  other  scenarios,  that  there  is  no  threshold  above  the 
background  level  of  particulate  matter. 

26See  Research  Priorities  for  Airborne  Particulate  Matter,  Vol.  1  (1998),  p.  19. 
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Committee  strongly  recommended  that  EPA  implement  a  targeted  research 
program  to  address  these  unanswered  questions  and  uncertainties.  In  its 
fiscal  year  1998  appropriations,  the  Congress  provided  funding  of  $49.6 
million  specifically  for  research  on  particulate  matter.  The  Congress  also 
directed  the  EPA  Administrator  to  arrange  for  an  independent  study  to, 
among  other  things,  monitor  and  report  on  progress  toward  improved 
understanding  of  the  relationship  between  particulate  matter  and  its  health 
effects.  The  National  Research  Council  is  carrying  out  this  mandated  work. 

Some  research  findings  have  recently  been  released  that  buttress  the 
findings  in  the  studies  EPA  relied  on  in  making  its  decisions  to  regulate  fine 
particles  in  1997.  For  example,  in  July  2000,  the  Health  Effects  Institute 
reported  that  its  reanalysis  and  additional  analysis  of  the  underlying  data 
essentially  validated  two  key  studies,  as  well  as  these  studies’  findings  of  an 
association  between  particulate  matter  and  mortality,  that  EPA  had  relied 
on  in  issuing  its  1997  rule.27 

Although  EPA  develops  what  it  considers  best  estimates  rather  than 
precautionary  estimates  of  the  risks  from  particulate  matter,  the  Clean  Air 
Act  requires  that  the  health-based  standards  be  set  with  an  “adequate 
margin  of  safety.”  According  to  officials  from  EPA’s  Office  of  Air  and 
Radiation,  building  in  the  required  margin  of  safety  for  the  particulate 
matter  standards  was  a  risk  management  decision,  not  part  of  the 
assessment  of  risk.  That  is,  the  margin  of  safety  was  incorporated  into  the 
allowable  concentration  levels,  or  standards,  chosen  by  the  Administrator. 
These  standards  have  been  challenged  in  court.  A  summary  of  the  issues 
associated  with  the  decision  over  the  levels  at  which  the  standards  were 
set,  as  well  as  the  court  case,  is  included  in  appendix  IV.  While  the  judicial 
review  of  the  1997  particulate  matter  standards  proceeds,  the  next  5-year 
mandated  review  of  the  standards,  expected  to  be  completed  in  2002,  is 
also  under  way. 


27The  Health  Effects  Institute  is  a  nonpartisan,  independent  research  organization  whose 
major  funding  is  from  EPA  and  the  auto  industry.  See  its  Reanalysis  of  the  Harvard  Six  Cities 
Study  and  the  American  Cancer  Society  Study  of  Particulate  Air  Pollution  and  Mortality  A 
Special  Report  of  the  Institutes  Particle  Epidemiology  Reanalysis  Project  (July  2000). 
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EPA’s  June  2000  Proposed 
National  Primary  Drinking 
Water  Regulation  for 
Arsenic 


Under  the  Safe  Drinking  Water  Act,  EPA  sets  health-based,  legally 
enforceable  drinking  water  standards  limiting  the  level  of  contaminants  in 
the  nation’s  drinking  water  systems  that  can  adversely  affect  public  health. 
First,  EPA  establishes  a  health-based  goal  at  a  level  at  which  no  known  or 
anticipated  adverse  health  effects  occur  and  that  allows  an  “adequate 
margin  of  safety.”  If  a  contaminant,  such  as  arsenic,  is  likely  to  cause 
cancer,  EPA  generally  sets  the  goal  at  zero.  After  setting  the  goal,  EPA 
typically  establishes  an  enforceable  standard,  called  a  maximum 
contaminant  level,  that  is  as  close  to  the  health-based  goal  as  is  feasible, 
considering  the  available  technology  and  costs.28  EPA  is  also  to  complete 
an  economic  analysis  to  determine  whether  the  benefits  of  the  standard 
justify  the  costs.  If  the  benefits  do  not  appear  to  be  justified,  EPA  may 
adjust  the  standard  to  a  level  that  “maximizes  health  risk  reduction  benefits 
at  a  cost  that  is  justified  by  the  benefits.”29  This  latter  provision  is  a  new 
authority  the  Congress  gave  to  EPA  under  amendments  to  the  Safe 
Drinking  Water  Act  in  1996. 


The  1996  amendments  to  the  Safe  Drinking  Water  Act  also  require  EPA  to 
propose  a  new  arsenic  standard  by  January  1,  2000,  and  to  issue  the  final 
rule  by  January  1,  2001,  Recognized  as  a  toxic  element  for  centuries, 
arsenic  has  also  been  associated  with  skin,  bladder,  and  other  cancers.30 
EPA  issued  its  proposed  arsenic  standard  in  June  2000.  Specifically,  EPA 
proposed  setting  the  maximum  contaminant  level  goal  at  zero  and  lowering 
the  enforceable  contaminant  standard  from  50  micrograms  per  liter  to  5 
micrograms  per  liter.31  EPA  is  using  its  new  authority  to  set  the  standard  at 
a  higher  level  than  the  technologically  feasible  level  of  3  micrograms  per 
liter  both  because  (1)  it  did  not  believe  that  the  costs  were  justified  by  the 
benefits  at  that  level  and  (2)  there  were  a  number  of  uncertainties, 
including  scientific  uncertainty  about  the  health  effects  of  arsenic  at  low 
levels  of  exposure  in  drinking  water.  EPA  also  requested  public  comment 
on  alternative  standards  of  3,  10,  and  20  micrograms  per  liter. 


28Drinking  water  standards  also  often  apply  to  contaminated  groundwater  at  hazardous 
waste  sites  regulated  under  EPA’s  Resource  Conservation  and  Recovery  Act  and  Superfund 
programs,  and  arsenic  is  a  key  contaminant  at  many  of  these  sites. 

29P.L.  104-182,  llOStat.  1613,  1624  (1996). 

30These  cancers  have  been  associated  with  arsenic  present  in  drinking  water  at 
concentrations  higher  than  those  observed  in  U.S.  drinking  water  supplies. 

31 A  microgram  is  one-millionth  of  a  gram.  The  Public  Health  Service  first  established  the 
standard  of  50  micrograms  per  liter  in  1942. 
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Arsenic  occurs  naturally  in  the  environment,  for  example,  in  rocks,  soil, 
and  groundwater,  and  has  been  used  in  a  variety  of  commercial  activities, 
most  notably  today  as  a  component  of  wood  preservatives.  Because 
arsenic  does  not  degrade  in  the  environment,  contamination  from 
historical  releases  is  cumulative.  EPA  estimates  that  about  10  percent  of 
groundwater  and  surface  water  systems  have  average  arsenic  levels  above 
5  micrograms  per  liter,  and  about  4.5  percent  have  average  arsenic  levels 
above  10  micrograms.  Arsenic  concentrations  in  groundwater  generally  are 
highest  in  the  Western  United  States,  where  EPA  estimates  that  12  percent 
of  groundwater  systems  exceed  10  micrograms  per  liter.  In  setting  this 
standard,  EPA  sought  to  protect  public  health  at  a  level  such  that  at  least  90 
percent  of  the  exposed  population  would  face  a  lifetime  risk  of  less  than  1 
in  10,000  for  developing  bladder  cancer.  EPA  estimates  that  the  new 
standard  will  avoid  16  to  36  bladder  cancer  cases  each  year,  4  to  9  of  which 
would  be  fatal.  As  discussed  below,  EPA  reported  that  several  other  health 
benefits  are  expected  to  result  from  the  proposed  standard,  including 
reductions  in  lung  cancer,  that  it  cannot  reliably  quantify  at  this  time. 

For  the  proposed  arsenic  standard,  EPA  estimated  the  health  risks  of 
bladder  cancer  on  the  basis  of  epidemiological  studies,32  relying  primarily 
on  a  review  of  arsenic  health  effects  research  conducted  by  the  National 
Research  Council  at  EPAs  request.  The  Safe  Drinking  Water  Act  requires 
EPA  to  consider  peer-reviewed  scientific  information  on  health  effects  in 
setting  drinking  water  standards,  and  EPA  considered  the  Council’s  study 
as  presenting  the  best  available  peer-reviewed  science.  EPA  used  four 
alternative  dose-response  relationships  reported  by  the  National  Research 
Council.  These  relationships  were  based  on  epidemiological  data  on 
bladder  cancer  mortality  in  a  high-arsenic  region  in  Taiwan.  The  median 
arsenic  concentrations  in  the  42  Taiwanese  villages  studied  ranged  from  10 
micrograms  per  liter  to  934  micrograms  per  liter;  29  of  the  villages  had 
median  arsenic  concentrations  at  or  above  100  micrograms  per  liter.  The 
National  Research  Council  noted  that  studies  in  Chile  and  Argentina 
observed  risks  of  lung  and  bladder  cancer  of  the  same  magnitude  as  those 
reported  in  Taiwan  at  comparable  levels  of  exposure. 

EPA’s  estimate  of  the  reductions  in  health  risks  and  the  benefits  of  its 
proposed  arsenic  standard  relied  on  the  following  key  precautionary 
assumptions: 


32 As  noted  earlier,  arsenic  is  one  of  the  few  contaminants  that  causes  cancer  in  humans  but 
not  in  laboratory  animals. 
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•  EPA  assumed  that  the  dose-response  relationship  for  arsenic  is  linear 
without  a  threshold,  even  at  low  concentrations,  rather  than  sublinear 
and/or  with  a  threshold.  (For  either  of  these  alternative  assumptions, 
the  response  at  low  doses  would  have  been  less  than  that  predicted  by  a 
linear  dose-response  relationship.)  EPA  made  this  assumption  because 
it  concluded  that  the  scientific  evidence  suggesting  a  sublinear  dose- 
response  relationship  is  not  strong  enough  to  depart  from  its  default 
assumption  of  a  linear,  no-threshold  dose-response  relationship.  While 
the  National  Research  Council  noted  that  the  most  plausible  scientific 
evidence  on  how  arsenic  causes  cancer  supports  a  sublinear  dose- 
response  relationship,  the  Council  concluded  that  the  available  evidence 
was  inconclusive  and  did  not  meet  EPA’s  stated  criteria  in  its  1996 
proposed  cancer  risk  assessment  guidelines  for  departure  from  the 
default  assumption  of  linearity.  In  its  proposed  rule,  EPA  solicited 
comments  on  this  issue. 

•  EPA  assumed  that  the  National  Research  Council’s  risk  estimates  based 
on  epidemiological  data  for  a  rural  population  in  Taiwan  could  apply  to 
the  U.S.  population,  even  though  the  Taiwanese  diet,  compared  to  the 
U.S.  diet,  includes  (1)  higher  levels  of  arsenic  and  (2)  lower  levels  of 
selenium,  which  has  been  shown  to  moderate  the  adverse  health  effects 
of  arsenic  in  animal  studies.  The  Council  noted  that  available  data 
suggest  that  arsenic  intake  from  food  is  higher  in  Taiwan  than  in  the 
United  States,  and  EPA  stated  in  its  proposed  rule  that  “arsenic  intake 
(by  persons  in  the  Taiwanese  study  region)  from  sources  other  than 
drinking  water  would  overestimate  the  unit  risk  calculated  from  the 
Taiwan  study.”  The  Council  also  noted  that  these  differences  “could 
affect  the  relevance  of  the  results”  of  its  Taiwanese-based  risk  estimates 
for  a  risk  assessment  for  the  U.S.  population.  In  light  of  this  concern,  the 
Council  recommended  that  EPA  investigate  the  relationship  between 
nutritional  factors  in  study  populations  and  susceptibility  to  arsenic- 
induced  cancer.  Without  this  information,  the  Council  indicated  that  it 
might  be  appropriate  to  be  precautionary  in  risk  assessments  of  arsenic. 

•  Because  of  data  limitations,  EPA  relied  on  the  National  Research 
Council’s  assumption  that  the  individuals  in  the  Taiwanese  study  who 
got  bladder  cancer  were  exposed  to  the  median  (50th  percentile)  level 
of  arsenic  found  in  the  water  of  their  villages’  wells.  In  its  notice  of  the 
proposed  standard,  EPA  acknowledged  that  this  assumption  is 
precautionary,  citing  its  Expert  Panel  on  Arsenic  Carcinogenicity,  which 
said  that  biases  from  using  average  doses  for  groups  lead  to 
overestimation  of  risk. 
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Each  of  these  assumptions  tends  to  overestimate  the  risk  of  arsenic 
exposure  and  the  benefits  associated  with  reducing  those  risks.  EPA’s 
proposed  rule  clearly  identifies  the  precautionary  assumptions  discussed 
above  and  also  indicates  that  the  agency  will  reexamine  the  use  of  these 
assumptions  before  issuing  the  final  rule.33  The  proposed  rule  also 
indicates  that  EPA’s  use  of  these  assumptions  may  have  resulted  in  a 
significant  overestimation  of  the  risk  of  bladder  cancer.  Consequently, 
EPA’s  estimate  of  the  related  benefits  may  also  be  significantly  overstated. 
EPA  officials  told  us  that  they  did  not  attempt  to  determine  the  effects  of 
varying  these  precautionary  assumptions  by  conducting  sensitivity 
analyses  because  they  did  not  have  sufficient  information  to  do  so.  We 
note,  however,  that  even  in  the  absence  of  sufficient  data,  sensitivity 
analyses  can  use  “what  if’  assumptions  to  assess  the  potential  impact  of 
precautionary  assumptions.  Furthermore,  EPA’s  guidelines  for  preparing 
economic  analyses  state  that  sensitivity  analyses  should  be  performed  on 
key  assumptions,  if  feasible,  in  assessing  and  presenting  uncertainty.34 
Finally,  EPA’s  cost-benefit  analysis  report  identified  the  precautionary 
assumption  of  a  linear  dose-response  relationship  but  did  not  identify  the 
other  precautionary  assumptions  discussed  above.35  Thus,  some  key 
scientific  uncertainties  associated  with  the  arsenic  benefits  estimates  are 
not  reflected  in  this  report. 


33EPA’s  proposed  rule  represents  the  agency’s  documentation  of  its  risk  assessment  on 
arsenic  at  this  time.  EPA  also  cites  the  report  of  the  National  Research  Council  from  which 
it  derived  its  risk  estimates,  Arsenic  in  Drinking  Water,  National  Academy  Press  (1999). 

34This  guidance  is  consistent  with  GAO’s  1997  recommendation  to  EPA  to  ensure  that  the 
agency’s  cost-benefit  analyses  identify  the  sensitivity  of  benefit  and  cost  estimates  when 
there  are  major  sources  of  uncertainty.  See  Air  Pollution:  Information  Contained  in  EPAs 
Regulatory  Impact  Analyses  Can  Be  Made  Clearer  (GAO/RCED-97-38,  Apr.  14, 1997).  We 
have  also  recommended  that  the  Office  of  Management  and  Budget  amend  its  guidance  to 
strengthen  the  clarity  and  credibility  of  cost-benefit  analyses.  See  Regulatory  Reform: 
Agencies  Could  Improve  Development,  Documentation,  and  Clarity  of  Regulatory  Economic 
Analyses  (GAO/RCED-98-142,  May  26,  1998). 

35 Proposed  Arsenic  in  Drinking  Water  Rule:  Regulatory  Impact  Analysis,  EPA  815-R-00-013 
(June  2000). 
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The  National  Research  Council  concluded  that  risk  assessments  of  bladder, 
lung,  and  other  cancers  associated  with  arsenic  can  be  performed  and  also 
recommended  further  study  to  characterize  a  number  of  adverse 
noncancer  effects,  including  heart  disease  and  diabetes.  Because  EPA 
based  its  risk  estimate  only  on  bladder  cancer,  the  agency  believes  that  it 
underestimated  the  combined  risk  of  all  arsenic-induced  health  effects.  In 
its  proposed  rule,  EPA  said  that  its  decision  to  use  only  bladder  cancer  was 
a  judgment  call  that  was  guided  by  the  best  available  science.  In  addition, 
EPA  did  develop  a  rough  estimate  of  the  benefits  of  reductions  in  lung 
cancer,  using  what  it  termed  a  “what  if’  scenario,  and  it  identified  them  as 
potential  benefits  in  its  cost-benefit  comparisons.36  EPA  expects  to  have  a 
peer-reviewed  quantitative  analysis  of  lung  cancer  risk  available  for  its  risk 
assessment  and  benefits  estimate  for  the  final  rule.  EPA’s  cost-benefit 
analysis  report  also  indicates  that  the  monetary  benefits  associated  with 
reductions  in  skin  cancer  would  be  minimal.  Finally,  EPA  does  not  believe 
it  has  sufficient  information  to  quantify  the  numerous  noncancer  health 
effects  associated  with  arsenic. 

Although  EPA’s  risk  assessment  and  benefits  estimate  included  several 
precautionary  assumptions,  EPA  also  used  new  analytical  techniques  that 
removed  other  precautionary  assumptions  used  by  the  agency  in  the  past:37 

•  According  to  officials  in  EPA’s  Office  of  Water,  past  risk  assessments  and 
benefits  estimates  for  drinking  water  standards  typically  relied  on  the 
precautionary  assumption  that  all  individuals  consume  2  liters  of 
drinking  water  per  day  over  his  or  her  lifetime,  which  is  more  than  the 
average  individual  is  believed  to  consume.  This  assumption  tended  to 
overestimate  risk  because  health  risks  from  contaminants  in  drinking 
water  increase  with  water  consumption.  For  the  proposed  arsenic  rule, 
however,  EPA  assumed  that  different  individuals  consume  different 
amounts  of  water.  EPA’s  estimates  of  these  varying  amounts  were  based 
on  recent  survey  data  indicating  that,  on  average,  an  individual 
consumes  only  about  1  liter  of  drinking  water  a  day.  By  using  more 


36For  its  rough  estimate  of  lung  cancer  benefits,  EPA  relied  on  the  National  Research 
Councils  statement  that  “some  studies  have  shown  that  excess  lung  cancer  deaths 
attributed  to  arsenic  are  2-  to  5-fold  greater  than  the  excess  bladder  cancer  deaths.”  EPA 
acknowledged  that  this  estimate  is  probably  too  high  because  a  reanalysis  of  the  studies 
indicates  that  the  excess  lung  cancer  deaths  attributed  to  arsenic  are  approximately  equal  to 
the  excess  bladder  cancer  deaths. 

37EPA  calculated  risks  using  Monte  Carlo  analysis,  which  employs  computer  simulations  to 
calculate  a  range  of  risk  values  rather  than  a  single  “point  estimate’  of  risk. 


Page  30 


GAO-Ol-55  EPA’s  Use  of  Precautionary  Assumptions 


realistic  water  consumption  assumptions,  EPA  produced  both  risk  and 
benefits  estimates  that  were  significantly  less  than  they  would  have 
been  if  it  had  continued  to  use  the  precautionary  assumption  about 
water  consumption. 

•  EPA’s  risk  assessment  and  benefits  estimate  for  arsenic  also  did  not 
include  another  significant  precautionary  assumption  that  EPA  typically 
uses  for  carcinogens.  Although  EPA  did  make  the  precautionary 
assumption  that  the  dose-response  relationship  is  linear  with  no 
threshold,  it  did  not  use  the  upper-bound  estimate  of  this  linear 
relationship.  Instead,  it  used  ranges  of  linear  dose-response 
relationships  derived  from  a  range  of  risk  estimates  reported  by  the 
National  Research  Council.  Because  EPA  used  ranges  of  dose-response 
relationships,  its  risk  estimates  and  benefits  estimates  were  significantly 
less  than  they  would  have  been  if  it  had  used  its  typical  precautionary 
assumption  of  an  upper-bound  dose-response  relationship. 


Conclusions 


The  particulate  matter  and  arsenic  rules  demonstrate  some  of  the  key 
uncertainties  that  EPA  must  address  in  evaluating  the  health  effects  of 
contaminants  and  determining  the  appropriate  actions  to  take.  In  terms  of 
the  types  of  precautionary  assumptions  that  EPA  may  use  to  address 
scientific  data  gaps  and  uncertainties,  these  cases  are  perhaps  atypical  in 
that  sufficient  epidemiological  data  are  available  in  both  cases  to  evaluate 
human  health  risks,  whereas  EPA  generally  must  extrapolate  from  animal 
data.  However,  they  are  quite  typical  in  that  their  fundamental  uncertainty 
is  about  adverse  health  effects  at  the  lowest  levels  of  exposure — 
specifically,  whether  there  is  a  threshold  below  which  adverse  health 
effects  do  not  occur.  Neither  epidemiological  nor  animal  data  can  provide 
complete  information  on  health  effects  at  the  low  levels  to  which  people 
are  typically  exposed,  and  therefore  assumptions  about  these  health  effects 
must  be  made.  Greater  uncertainty  about  the  effects  at  low  levels  may  exist 
for  arsenic  than  for  particulate  matter  because  the  epidemiological 
information  on  particulate  matter  is  based  on  actual  exposure  levels  in  the 
United  States,  whereas  the  data  on  arsenic  are  primarily  from  Taiwanese 
populations  exposed  to  much  higher  levels  of  arsenic  than  U.S. 
populations.  However,  the  case  of  particulate  matter  illustrates  how 
difficult  it  is  to  scientifically  resolve  uncertainties  about  causality 
associated  with  the  use  of  epidemiological  data — how  challenging  it  is  to 
conclusively  demonstrate  that  the  health  effects  associated  with 
particulate  matter  are,  in  fact,  primarily  caused  by  exposure  to  the 
particulate  matter  rather  than  other  factors,  including  other  pollutants. 
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While  EPA  identified  in  its  proposed  regulation  the  key  health  uncertainties 
and  precautionary  assumptions  it  used  in  assessing  the  risks  of  arsenic  in 
drinking  water,  it  did  not  identify  all  of  them  in  its  formal  cost-benefit 
analysis.  Because  this  regulation  is  still  being  developed,  EPA  has  the 
opportunity  both  to  fully  disclose  the  precautionary  assumptions  used  and 
to  incorporate  some  additional  analyses  into  its  final  regulation.  For 
example,  sensitivity  analyses  showing  the  impact  of  precautionary 
assumptions  on  the  risk  assessment  and  benefits  estimate  could  help  the 
Administrator  in  setting  the  standard  and  could  also  provide  other 
interested  parties  with  a  more  complete  understanding  of  the  potential 
range  of  benefits  of  the  standard. 

Recommendation  for 
Executive  Action 

To  better  inform  decisionmakers  about  the  effects  of  precautionary  health 
assumptions  on  EPA’s  estimates  of  the  benefits  associated  with  the  arsenic 
rule,  we  recommend  that  the  EPA  Administrator  ensure  that  EPA’s  cost- 
benefit  report  for  the  final  rule  fully  disclose  the  precautionary 
assumptions  used  and  provide  sensitivity  analysis  on  the  key  precautionary 
assumptions  included  in  the  agency’s  benefits  estimate. 

Agency  Comments 

i 

We  provided  a  draft  of  this  report  to  EPA  for  review  and  comment.  In 
commenting  on  the  draft,  officials  from  the  National  Center  for 
Environmental  Assessment,  Office  of  Research  and  Development,  stated 
that  the  report  presents  a  fair  and  informative  discussion  of  the 
characteristics  of  data  typically  available  for  risk  assessment.  In  addition, 
the  officials  supported  the  recommendation  that  the  cost-benefit  report  for 
the  arsenic  rule  disclose  precautionary  assumptions  and  provide  sensitivity 
analysis,  subject  to  the  availability  of  data  pertinent  to  such  an  analysis. 
Officials  from  the  Offices  of  Air  and  Radiation;  Water;  and  Policy, 
Economics,  and  Innovation  provided  technical  comments  and 
clarifications  that  we  incorporated  as  appropriate. 

The  scope  and  methodology  for  our  work  are  discussed  in  appendix  I.  We 
conducted  our  work  from  October  1999  through  October  2000  in 
accordance  with  generally  accepted  government  auditing  standards. 

We  will  send  copies  of  this  report  to  the  Honorable  Carol  M.  Browner, 
Administrator,  EPA,  and  to  other  interested  parties.  We  will  also  make 
copies  available  to  others  on  request.  Please  call  me  or  Christine  Fishkin  at 
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(202)  512-6111  if  you  or  your  staff  have  any  questions.  Other  key 
contributors  to  this  report  are  David  Goldstein,  Bruce  Skud,  and  Susan 
Swearingen. 

David  G.  Wood 

DfcvicC  U)ouc2 


Director,  Natural  Resources  and 
Environment 
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Appendix  I _ 

Objectives,  Scope,  and  Methodology 


Objectives 


Scope 


The  Honorable  Bud  Shuster,  Chairman  of  the  House  Committee  on 
Transportation  and  Infrastructure,  and  Senator  Frank  R.  Lautenberg  asked 
us  to  examine  whether  the  Environmental  Protection  Agency’s  (EPA) 
benefits  estimates  for  major  environmental  regulations  that  establish 
health-based  standards  reflect  precautionary  assumptions  about  health 
risks.  As  agreed  with  our  requesters,  we  focused  our  work  on  (1)  key 
factors  that  affect  EPA’s  use  of  precautionary  assumptions  in  assessing 
health  risks  and  (2)  whether  EPA  used  and  identified  precautionary 
assumptions  in  estimating  the  health  risks  and  benefits  of  recent  major 
regulations  setting  health-based  standards. 


The  issue  of  whether  EPA’s  estimates  of  the  benefits  of  health-based 
environmental  standards  reflect  precautionary  health  risk  assumptions  is 
relevant  to  major  regulations  that  set  health-based  standards  and  estimate 
health  benefits.  The  benefits  of  major  regulations — that  include  those  with 
an  impact  on  the  economy  of  $100  million  or  more — are  provided  in 
economic  analyses  (also  referred  to  as  regulatory  impact  analyses) 
required  by  Executive  Order  12866.  In  fiscal  years  1997  through  1999,  four 
of  EPA’s  major  regulations  set  new  health-based  standards — two  each 
under  the  Clean  Air  Act  and  the  Safe  Drinking  Water  Act.  In  fiscal  year  2000 
through  June  2000,  EPA  proposed  two  health-based  standards  under  the 
Safe  Drinking  Water  Act.  We  selected  one  rule  each  from  EPA’s  air  and 
water  programs  in  order  to  report  on  the  agency’s  current  methodologies 
and  practices  for  estimating  health  risks  and  benefits:  EPA’s  regulations  (1) 
establishing  air  quality  standards  for  particulate  matter  in  July  1997  and  (2) 
proposing  a  drinking  water  standard  for  arsenic  in  June  2000. 

These  regulations  generally  reflect  EPA’s  current  practices  and  procedures 
for  risk  assessments  and  benefits  estimates  for  major  categories  of 
contaminants  regulated  by  EPA  under  the  Clean  Air  Act  and  the  Safe 
Drinking  Water  Act.  For  example,  the  particulate  matter  rule  covers  one  of 
the  six  widespread  air  pollutants  that  EPA  is  required  to  set  standards  for 
and  to  review  every  5  years.  EPA  uses  similar  approaches  in  estimating 
health  risks  and  health  benefits  for  the  other  criteria  pollutants,  including 
the  ozone  rule  that  was  also  promulgated  in  1997.  As  of  June  2000,  EPA  had 
not  proposed  or  finalized  other  health-based  standards  for  criteria 
pollutants  since  1997.  Similarly,  we  reviewed  EPA’s  proposed  rule  to  revise 
standards  for  arsenic  in  drinking  water  under  the  Safe  Drinking  Water  Act, 
issued  for  comment  in  June  2000,  because  it  incorporates  new  approaches 
to  setting  health-based  standards  and  estimating  benefits  reflecting  the 
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requirements  of  the  1996  amendments  to  the  Safe  Drinking  Water  Act.  This 
rule  incorporates  the  general  approach  the  agency  plans  to  use  in  setting 
health-based  standards  under  the  act. 

Although  these  regulations  provide  examples  of  the  differing  extent  to 
which  EPA  may  use  precautionary  assumptions  in  risk  assessments  and 
benefits  estimates,  the  results  may  not  be  generalizable  to  all  of  EPA’s 
major  regulations  setting  health-based  standards.  In  addition,  our  review  of 
the  benefits  estimates  is  limited  to  examining  the  extent  to  which  EPA  used 
precautionary  assumptions  in  assessing  health  risks  and  in  estimating  the 
benefits  of  the  proposed  or  final  health-based  standards.  As  a  result,  we  did 
not  review  the  economic  aspects  of  EPA’s  benefits  estimates,  such  as  the 
methods  used  to  estimate  dollar  values  for  lives  saved  and  illnesses 
avoided.  Finally,  we  did  not  assess  the  extent  to  which  EPA’s  benefits 
estimates  comply  with  the  agency’s  and  the  Office  of  Management  and 
Budget’s  guidance  for  preparing  economic  analyses  of  significant 
regulatory  actions,  which  include  guidance  on  the  treatment  of  risk  and 
uncertainty  and  reporting  the  “best”  or  most  likely  estimate. 


Methodology 


To  determine  the  key  factors  that  affect  EPA’s  use  of  precautionary 
assumptions  in  assessing  health  risks,  we  reviewed  EPA  statutes  that 
address  health-based  standards,  EPA  policy  guidance  on  risk  assessment 
and  risk  characterization,  and  relevant  reports  by  the  National  Academy  of 
Science’s  National  Research  Council  and  the  Presidential/Congressional 
Commission  on  Risk  Assessment  and  Risk  Management.  We  also 
interviewed  officials  in  EPA’s  Offices  of  Policy,  Economics,  and  Innovation; 
Water;  Air  and  Radiation;  and  Research  and  Development.  To  obtain 
perspectives  on  the  use  of  precautionary  assumptions  in  health  risk 
assessment,  we  interviewed  experts  from  the  Health  Effects  Institute,  the 
Harvard  School  of  Public  Health,  Resources  for  the  Future,  and  an  official 
of  the  American  Industrial  Health  Council.  We  also  reviewed  a  variety  of 
scientific  articles  on  health-based  risk  assessment.  Finally,  we  attended 
risk  assessment  and  environmental  regulation  seminars  at  the  Harvard 
School  of  Public  Health’s  Center  for  Risk  Analysis,  from  which  we  obtained 
information  incorporated  into  this  report. 

To  determine  how  precautionary  assumptions  affected  the  health-based 
standards  and  the  estimated  benefits  of  recent  major  regulations  and 
whether  EPA  clearly  identified  these  assumptions,  we  reviewed  (1)  the 
proposed  and  final  rules  for  particulate  matter,  the  quantitative  health  risk 
assessment,  the  regulatory  impact  analysis,  and  other  key  documents, 
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including  EPA’s  criteria  document  on  particulate  matter  and  recent  judicial 
findings  on  the  particulate  matter  rule  and  (2)  the  proposed  rule  for 
arsenic,  regulatory  impact  analysis,  and  other  supporting  documents, 
including  the  1999  report,  Arsenic  in  Drinking  Water,  by  the  National 
Research  Council  that  EPA  relied  on  for  its  risk  assessment.  We  attended 
the  June  2000  meeting  of  EPAs  Science  Advisory  Board  on  the  proposed 
arsenic  regulation.  We  met  with  officials  in  the  air  and  water  program 
offices  to  obtain,  among  other  information,  their  views  on  the  use  of 
precautionary  assumptions  in  these  rules. 
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Dose-Response  Relationships  Based  on 
Varying  Assumptions 


In  assessing  cancer  risks,  EPA  generally  assumes  a  linear,  no-threshold 
relationship  between  the  dose  of  a  contaminant  and  its  health  effect  at  low 
doses.  This  assumption  posits  that  (1)  dose  and  health  effect  vary  linearly, 
that  is,  if  a  dose  is  doubled,  the  health  effect  doubles;  and  (2)  there  is  no 
safe  level,  or  threshold,  of  exposure  to  a  carcinogen— that  is,  any  exposure 
presents  some  risk  of  developing  cancer.  However,  for  many  contaminants, 
a  threshold  and/or  a  sublinear  dose-response  relationship  is  believed  to  be 
plausible.  Both  sublinear  and  threshold  dose-response  relationships 
typically  lead  to  lower  risk  estimates  at  the  lowest  dose  levels  than  does  a 
linear,  no-threshold  dose-response  relationship.1  These  varying  dose- 
response  relationships  are  shown  in  figure  3. 


Figure  3:  Dose-Response  Relationships  Based  on  Varying  Assumptions 


Increasing  response  (health  risk) 


Linear,  no-threshold 
—  —  Threshold 

-  Sublinear 


‘According  to  EPA  officials,  some  contaminants,  such  as  vinyl  chloride,  pose  a  higher  risk  at 
low  doses  than  that  predicted  by  a  linear  dose-response  relationship.  The  dose-response 
relationship  for  such  contaminants  is  referred  to  as  supralinear. 
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Summary  of  EPA’s  1997  Particulate  Matter 
Standards 


In  July  1997,  EPA  established  its  most  recent  National  Ambient  Air  Quality 
Standards  for  particulate  matter  (commonly  called  soot),  which  included 
new  standards  for  the  fine  fraction  of  particulate  matter,  also  known  as  fine 
particulate  matter.1  The  standards  are  for  annual  and  daily  concentrations 
of  particulate  matter  measured  at  various  locations  across  the  United 
States.  The  annual  standards  are  intended  to  provide  protection  against 
typical  day-to-day  exposures  as  well  as  longer-term  exposures.  The  daily 
standards  are  intended  to  provide  protection  against  days  with  high  peak 
concentrations  of  particulate  matter,  localized  “hot  spots,”  and  risks  from 
seasonal  emissions  that  would  not  be  well  controlled  by  national  annual 
standards. 


Table  1 :  1997  Particulate  Matter  Standards 


Standards  in  micrograms  per  cubic  meter 

Type  of  particulate  matter 

Annual  standard 

24-hour  (daily)  standard 

Particulate  matter  (PM10)a 

50b 

150c 

Fine  particulate  matter 

15d 

65e 

(PM2.5) 

includes  both  coarse  and  fine  particles. 

bBased  on  the  3-year  average  of  annual  arithmetic  mean  concentrations  of  PM  1 0  at  specified  monitors. 

cBased  on  the  3-year  average  of  the  99th  percentile  of  24-hour  concentrations  of  PM  10  at  specified 
monitors. 

dBased  on  the  3-year  average  of  annual  arithmetic  mean  concentrations  of  PM2.5  at  specified 
monitors. 

eBased  on  the  3-year  average  of  the  98th  percentile  of  24-hour  concentrations  of  PM2.5  at  specified 
monitors. 


These  are  the  primary  standards,  intended  to  protect  the  public  health.  The  Clean  Air  Act 
also  calls  for  secondary  standards,  set  at  a  level  to  protect  the  public  welfare,  e.g.,  to 
address  effects  on  visibility,  vegetation,  and  wildlife. 
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Issues  Associated  With  EPA’s  Setting  of  the 
1997  Particulate  Matter  Standards 


Although  EPA  develops  what  it  considers  best  estimates  rather  than 
precautionary  estimates  of  the  risks  from  particulate  matter,  the  Clean  Air 
Act  requires  that  the  health-based  standards  be  set  with  an  “adequate 
margin  of  safety.”  According  to  EPA,  the  margin  of  safety  was  incorporated 
into  the  allowable  concentration  levels,  or  standards,  chosen  by  the 
Administrator.  Under  EPA’s  interpretation  of  the  Clean  Air  Act,  the  agency 
is  not  required  to  set  air  quality  standards  at  a  zero-risk  level  to  achieve  an 
adequate  margin  of  safety,  but  simply  at  a  level  that  avoids  unacceptable 
risks.  For  example,  to  build  in  this  margin  of  safety  for  the  standard  for  the 
allowable  annual  concentration  of  particulate  matter  (also  called  the 
“annual  standard"),  the  Administrator  chose  15  micrograms  of  particulate 
matter  per  cubic  meter  of  air. 

This  concentration  was  somewhat  lower-— that  is,  it  was  more  protective  of 
health— than  the  concentration  at  which  the  epidemiological  evidence  of 
adverse  health  effects,  particularly  mortality,  was  most  consistent  and 
coherent.  For  example,  several  key  epidemiological  studies  reported 
statistically  significant  associations  between  mortality  and  illness  and 
annual  particulate  matter  concentrations  of  about  16  to  about  21 
micrograms  per  cubic  meter  of  air.  According  to  EPA,  even  differences  of  1 
microgram  per  cubic  meter  of  air  in  the  annual  concentration  in  this  range, 
such  as  the  difference  between  16  and  15,  can  significantly  affect  health 
risks.  The  Administrator  believed  that  an  annual  standard  of  15  micrograms 
per  cubic  meter  of  air  would  provide  an  adequate  margin  of  safety  against 
the  health  problems  reported  in  the  scientific  literature.  Although  some 
studies  did  show  health  effects  at  lower  levels,  EPA  believed  that  the 
scientific  uncertainties  about  the  health  effects  at  such  levels  were  too 
great  to  support  standards  at  those  concentrations. 
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Issues  Associated  With  EPA’s  Setting  of  the 
1997  Particulate  Matter  Standards 


As  soon  as  the  particulate  matter  standards  were  promulgated  in  July  1997, 
they  were  challenged  in  court  by  industry  representatives,  small 
businesses,  and  some  states,  which  were  concerned  about  the  potential 
economic  impact  of  the  stricter  standards,  among  other  things.  Many  of 
these  parties  had  fought  the  issuance  of  the  standards.  Critics  had  argued, 
for  example,  that  the  costs  of  implementing  the  standards  could  run  as  high 
as  $46  billion  per  year  and  would  cause  serious  financial  harm  to  key 
segments  of  the  U.S.  economy  without  providing  significant  health 
benefits.  The  parties  challenged  the  rule  on  a  variety  of  grounds,  including 
questions  about  the  scientific  uncertainties  associated  with  the  health 
effects  of  particulate  matter.  In  May  1999,  a  U.S.  Court  of  Appeals 
remanded  the  particulate  matter  standards,  as  well  as  the  ozone  standards 
issued  at  the  same  time,  to  EPA  for  further  consideration.1  EPA  appealed 
this  ruling  to  the  Supreme  Court,  and  in  May  2000,  the  Court  agreed  to 
review  the  case.  Oral  arguments  in  the  case  are  scheduled  for  November 
2000. 

A  key  concern  expressed  by  the  appeals  court  in  its  ruling  remanding  the 
standards  dealt  with  EPA’s  rationale  for  setting  the  allowable  annual 
concentration  of  particulate  matter  at  15  micrograms  per  cubic  meter  of 
air.  The  court  found  that  the  factors  EPA  used  in  determining  the  degree  of 
public  health  concerns  associated  with  different  concentrations  of 
particulate  matter  were  reasonable,  but  that  neither  the  Clean  Air  Act  nor 
EPA  had  articulated  an  “intelligible  principle,”  or  definitive  criterion,  to 
direct  how  the  agency  would  apply  these  factors.  Specifically,  for 
particulate  matter,  the  court  indicated  that  EPA  had  not  adequately 
articulated  how  a  standard  of  15  micrograms  per  cubic  meter  could  meet 
the  Clean  Air  Act’s  requirement  to  set  National  Ambient  Air  Quality 
Standards  with  a  margin  of  safety  when  it  is  also  EPA’s  judgment  that 
particulate  matter  poses  health  risks  at  any  level  above  zero.  Because  no 
intelligible  principle  was  articulated,  the  court  held  that  EPA’s  construction 
of  the  Clean  Air  Act  in  setting  the  particulate  matter  and  ozone  standards 
resulted  in  an  unconstitutional  delegation  of  legislative  authority.  Shortly 
after  the  Supreme  Court  agreed  to  review  the  case,  it  also  agreed  to 
consider  an  industry  petition  arguing  that  EPA  could  provide  the  intelligible 
principle  needed  for  setting  the  air  quality  standards,  at  least  in  part,  by 
taking  the  costs  of  implementing  the  regulations  into  consideration  when 
setting  the  standards.  While  the  judicial  review  of  the  1997  particulate 


1 American  Trucking  Associations  v.  EPA,  175  F.3d  1027  (D.C.  Cir.  1999). 
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Issues  Associated  With  EPA’s  Setting  of  the 
1997  Particulate  Matter  Standards 


matter  standards  proceeds,  the  next  5-year  mandated  review  of  the 
standards,  expected  to  be  completed  in  2002,  is  also  under  way. 
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